GUITESTER: A Log-Based Usability Testing Tool for Graphical User Interfaces

In this paper, we propose methods for testing the usability of graphical user interface (GUI) applications based on log files of user interactions. Log analysis by existing methods is not efficient because evaluators analyze a single log file or log files of the same user and then manually compare results. The methods proposed here solve this problem; the methods enable evaluators to analyze the log files of multiple users together by detecting interaction patterns that commonly appear in the log files. To achieve the methods, we first clarify usability attributes that can be evaluated by a log-based usability testing method and user interaction patterns that have to be detected for the evaluation. Based on an investigation on the information that can be obtained from the log files, we extract the attributes of clarity, safety, simplicity, and continuity. For the evaluations of clarity and safety, the interaction patterns that have to be detected include those from user errors. We then propose our methods for detecting interaction patterns from the log files of multiple users. Patterns that commonly appear in the log files are detected by utilizing a repeating pattern detection algorithm. By regarding an operation sequence recorded in a log file as a string and concatenating strings, common patterns are able to be detected as repeating patterns in the concatenated string. We next describe the implementation of the methods in a computer tool for logbased usability testing. The tool, GUITESTER, records userapplication interactions into log files, generates usability analysis data from the log files by applying the proposed methods, and visualizes the generated usability analysis data. To show the effectiveness of GUITESTER in finding usability problems, we report an example of a usability test. In this test, evaluators could find 14 problems in a tested GUI application. We finally discuss the ability of the proposed methods in terms of its log analysis efficiency, by comparing the analysis/sequence time (AT/ST) ratio of GUITESTER with those of other methods and tools. The ratio of GUITESTER is found to be smaller. This indicates the methods make log analysis more efficient. key words: usability, graphical user interfaces, humancomputer interaction, log files, interaction patterns